THE CEA FORUM 


Summer/Fall 

2010 


A CEA Forum Roundtable 

No Longer a Spinning Satellite: Designing a 
Collaborative Faculty Community through 
Qualitative Assessment 

KATIE TALERICO 

Art Institute of Pittsburgh 


Faced with the task of a department-wide assessment, I think I had a reaction not unlike others. 

In fact, during the assessment workshop at the New Orleans College Composition and 
Communication Conference during April 2008, the opening speaker echoed the words of 
Melville’s Bartleby, “I would prefer not to.” Having a penchant for the dramatic, I preferred to 
echo those final words of Kurtz from Heart of Darkness, “The Horror, the Horror!” 

Despite the arguments that had been made that assessment is not meant to evaluate a 
single instructor, take away from academic freedom, and signal the birth of universal curriculum, 
I couldn’t help but wonder, yes, but if the assessment goes badly, who will be to blame? Didn’t I 
enter this profession because I liked the idea of having the freedom to walk into a classroom, 
close my door, and teach according to my pedagogical beliefs? This may be an immature look at 
assessment, but I doubt that I am alone in my thoughts. 
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Our initial phase was a quantitative approach, looking at 30 percent of the portfolios of 
our ENG1010 courses and evaluating them against a rubric written collectively by the English 
faculty. While I am not arguing the benefits or uses of quantitative and qualitative assessment, 
something my colleagues discuss at length, I was interested in the effect that assessment had on 
us as instructors. 

Our first go-round with quantitative assessment took place during a Thursday in-service 
day. We decided to shut ourselves in a classroom with portfolios in hand and allotted the entire 
eight-hour day to working through the mountainous stack of portfolios that lay in the center of 
our circle. Because of the nature of quantitative assessment that we conducted, individuals 
reading and filling out rubrics for each assigned portfolio, silence was necessary. There we sat, 
surrounded by literally thousands of pages of text, pencils in hands. What happened next was the 
most natural thing, something that should occur when a group of composition instructors are 
given papers—we wanted to talk. All of us are, after all, in a long-term commitment with the 
field of composition. For better or worse, till death do us part, we are actively invested in helping 
writers learn how to express themselves more effectively. Give us a set of papers, and we will 
immediately find points to discuss. However, this inclination to talk only slowed down the 
process. We would come across a real gem in our piles and want to share it. “Wow! Look at the 
way this writer moves into the topic!” Words would suddenly erupt, bubble up like lively fresh 
water in a stream until we remembered, or were reminded, that we had many portfolios to get 
through and time was of the essence. 
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We were literally and metaphorically silenced - allowed only to put our ideas in the 
forms of checkmarks beside categories from the rubric: “Evidence of Process,” Check 3... 
“Demonstration of Intertextuality,” Check 2. It was a totally isolating experience. 

We felt angry at each other for talking, breathing, clicking a pencil, teaching so many 
students, taking too long on each paper, spending too little time on the paper. I can remember we 
even argued about lunch breaks—some wanted to get through the assessment and forgo a break. 
Others needed time to get away. Even my suggestion of a two-hour, four-martini lunch was 
refused. 

When we returned and watched the 4:30 dismissal hour arrive, it was like reliving 
detention in high school all over again. You mean we have to stay late! But we did. We stayed, 
and read, and marked, and read, and marked and finally went home. 

The end of the day came, and the portfolios were all assessed. Weeks later, numbers were 
crunched, and we took the task of looking at them to see what we could leam about our 
program’s development. However, the numbers themselves felt hollow—empty of the rich 
comments we had wanted to make but couldn’t. They pointed to changes we should consider 
making, but we felt a false confidence in their limited information. We realized they only 
answered the questions we asked, not the ones we didn’t ask, the ones that we hadn’t thought to 
probe. 

Given our frustration with quantitative assessment, as Jeff explains, we were finally given 
the green light to explore a form of qualitative assessment. Let me say that we were told up front 
we would most likely not be paid for our time, and many of us were already carrying five or six 
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classes per quarter. And no, a quarter system does not allow for summers off, so most of our time 
devoted to this project was evenings and weekends when we would return to our building to 
meet. 

Krystia Nora came into our group full of energy and excitement about assessment, and 
she warned us that qualitative assessment would require more time and work from us. The idea 
of more time did put us off a bit, but we were really looking for something more enriching and 
meaningful, and decided as a group that we were willing to try just about anything. She 
introduced us to the wide worlds of assessment, qualitative assessment, and qualitative research 
in order to let us choose a theoretical model and methodology together, as she describes in her 
section. In short, we selected a postmodern approach that centered around one question, how we 
grade, but also left us open to exploring other questions as they emerged. Almost immediately 
we witnessed a dramatic change in the way the group began to function. 

We, quite literally, had to talk. 

Our methodology had us sitting in the writers’ center again, as we had for our rubric 
assessment, but now we sat in a circle. It was our conversation that would be later used as the 
fertile ground to gather the assessment results. No longer silenced into our corners and forced to 
quietly read, we began our sessions talking amongst ourselves, often on unrelated topics. We 
joked; we laughed; we enjoyed the time together. But more important even than this, our 
conversation was not based around a rubric. Instead we really allowed ourselves permission to 
follow whatever theme came up in our conversations. We had a vague notion we would spend 
most of our time talking about grades, but we often ended up looking at widely different issues. 
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We also allowed the questions to emerge rather than setting up conclusions to reach. This shows 
how we were influenced by our readings of grounded theory, even if we did not explicitly 
confine ourselves to this approach. As described by Kathy Charmaz, “Grounded theorists 
evaluate the fit between their initial research interest and their emerging data. They do not force 
preconceived ideas and theories directly upon their data. Rather, they follow the leads that they 
define in the data, or design another way of collecting data to try to follow their initial 
interests” (250). The idea of us following our own leads and allowing ourselves permission to 
talk to the data and to ourselves was one that appealed to me. 

I think another fundamental difference was how we saw the conversations lead us back 
into new research. We would meet, talk, tape, analyze, assess, and then go out and find what 
others had to say on issues that came up. It led to not only assessment of the papers in our hands, 
but our pedagogies, practices, and department as a whole. Since these were questions and issues 
that arose out of our own conversations, it also didn’t feel like it was threatening. We no longer 
used numbers to judge each other (why is it that this department has low scores with Process 
Writing?), but instead we were forced to challenge our assumptions and beliefs. The most 
interesting papers to look at became not the passing ones, but the failing. We asked, “Why did 
this paper fail?” We questioned one another about the energy of the student and allowed 
ourselves to question what more we could have done to help the writer. 

Even when it came to coding the transcripts of our conversations, which initially felt like 
a daunting (and again isolating) task, we felt that we had pennission to bring our own energy to 
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the process. Again quoting Kathy Charmaz: “The codes emerge as you study your data. By 
studying your data, you again interact with it and ask questions of it” (245). 

I think one of the reasons we were so comfortable and open to this as a process was 
because of how it paralleled the way we taught students to write. Composition instructors often 
ask students to freewrite, to get as much of their ideas onto the paper and then allow instinct to 
take over later. In other words, let the argument emerge, much like we let the patterns and codes 
emerge. As Peter Elbow suggests in his seminal book, Writing Without Teachers, when talking 
about how to put ideas together after such a session of freewriting, writers might “find each other 
by feel, by drift, by intuition, by mindlessness” (63). So too with our discussions and later 
coding, we allowed ourselves to drift, to trust intuition, to be both mindless and purposeful in our 
attempt to discover new ways to improve how we teach student writers. At this point, we realized 
that our pedagogy in teaching writing paralleled this new assessment model. 

While our move to qualitative assessment may not seem significant (again I started not 
only apathetic but hostile in my approach to assessment), it really did put enthusiasm back into 
our group. Before we had cringed to stay 10 minutes past quitting time; now we were coming in 
early on Saturday mornings to voluntarily complete work. The group, no longer at odds with the 
approach and really believing in what we were doing, also became a cohesive unit where we 
supported and encouraged each other. We found that assessment could be something that not 
only showed us what we were doing in our classes, but also was an enriching experience that 
opened up issues in teaching and writing we had not sought out before. 
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